fix(review): bind verified evidence in scripts/ci/opencode_review_normalize_output.py#563
fix(review): bind verified evidence in scripts/ci/opencode_review_normalize_output.py#563seonghobae wants to merge 5 commits into
Conversation
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head4ff50eca109bb1410275e024c92fef348f278147. -
Head SHA:
4ff50eca109bb1410275e024c92fef348f278147 -
Workflow run: 29404144238
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["CI script: opencode_review_normalize_output.py"]
S1 --> I1["review and security gate shell path"]
I1 --> R1["Review risk: CI script: opencode_review_normalize_output.py"]
R1 --> V1["bash -n plus Strix self-test"]
Evidence --> S2["Test (2 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (2 files)"]
R2 --> V2["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["CI script: opencode_review_normalize_output.py"]
S1 --> I1["review and security gate shell path"]
I1 --> R1["Review risk: CI script: opencode_review_normalize_output.py"]
R1 --> V1["bash -n plus Strix self-test"]
Evidence --> S2["Test (2 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (2 files)"]
R2 --> V2["targeted test run"]
|
seonghobae
changed the title
seonghobae
changed the title
1 participant
Critical output contract observed on the protected base
The protected base normalizer accepts each adversarial probe only when the free-form evidence literally repeats the exact structured path and positive line as path:line. For every probe, write evidence in this exact shape: scripts/ci/opencode_review_normalize_output.py:699 observed RESULT source-line-sha256=64-lowercase-hex.
Purpose
Current-head runs proved that GPT-4.1 completes the structured review in about 31 seconds, but the protected base rejects it only because the evidence prose omits the duplicate path:positive-line string even when the structured path, line, observed outcome, and source receipt are otherwise present.
This PR minimally repairs that legacy binding. It adds the structured path:line only when the path is a sealed current-head changed file, the positive line exists, exactly one receipt is present, the receipt already matches the exact sealed line bytes, and independent proof plus an observed outcome are present.
The following remain fail-closed:
Current HEAD validation
Verified current-head source traces:
After this PR merges, PR #564 activates 30-minute small, 60-minute medium, and 90-minute large, unknown, and central per-model review timeouts.